There have been many attempts and approaches to increase the perceived realism of multi-media presentations. One method of increasing the viewers perceived realism has been to display an image of spherical coverage such that the viewer perceives that the scene surrounds him. To date there have been only limited attempts at recording, processing, and displaying an image of spherical coverage. A limitation of prior art systems was that no compact method existed, such as a helmet mounted display (HMD) system, for simulating the effect of viewing a scene of spherical coverage in a large display room. Such a system would be extremely useful and economical in post production use and for home viewing. All attempts have had limitations which the present invention attempts to overcome.
In recent years a variety of portable video camera and recorder systems have been integrated into a single unit popularly known as "camcorders". U.S. Pat. No. 4,571,628 by Thornton of Miami, Florida includes a binocular arrangement which include typical components found in conventional camcorders. In Thornton's assembly an adjustable object lens assembly observes the predetermined field of viewing and additional processing assemblies convert the resulting visual image into electrical video signals which are transferred to a remotely located video recorder which in turn transfers the process signal back to a liquid crystal display specifically dimensioned for mounting within the portable housing along independent lines of sight of the viewing assembly. Remote control means may serve to operate the video recorder so as to accomplish selective and substantially concurrent playback of the previously recorded field of vision selectively observed by the object lens assembly wherein such replay may be reviewed on the liquid crystal display structure within the housing. No portable video camera and recorder system proposed in prior art has included a lens assembly with spherical coverage.
Various systems have been proposed in which a transmitter transmits conventional TV signals over-the-air to a receiver located at a remote location. Radio Electronics, February 1986, p. 51+ entitled "Wireless Video Camera Link" describes such a system . No prior art has disclosed a method for transmitting and receiving a camera recorded scene of spherical coverage.
Starting in 1985 a large number of companies started marketing arrays of CRT-based (cathode ray tube) video monitors mounted in a variety of matrix geometries, such as four monitors by four monitors or 4.times.4 (whose overall image may be 6.6 feet by 5.0 feet), 3.times.3, 2.times.2, and also arrays involving as may as 356 monitors. The audience directly views directly the front faces of video monitors. These video array systems are capable of creating multi-image effects involving the entire matrix of CRT's using electronic digital image processing circuits. The user may present a single image covering the entire array or the same image on each individual CRT, or any combination of image/CRT displays. Examples of such video array systems are as follows: The"Vidiwall" (TM) of Philips of Holland, which is driven by a laser videodisk system using a large number of laser videodisks; a system marketed by TC Studios of West Germany, whose video array is driven by a 3/4" VTR (video tape recorder); a system marketed by Nurnberger Medientechnick GmbH (Tele-wall Delcom 256 Model) of West Germany, also operated by 3/4" VTR or laser disks; systems marketed by Electronic ("videowall" TM) of Minneapolis, MN. Electronic markets video systems which are both 3/4" VTR or laser video disk sources; and systems marketed by Video Matrix Corporation, U.S. Pat. No. 4,734,779 by Levis et al., whose system accepts a standard VTR/VCR or computer source and includes rear projection elements to hide the seams between display monitors. Furthermore, there are several custom-made systems in various places, mainly discotheques, around the United States and Europe. While a matrix of monitors have been arranged to form a wall of display monitors, the monitors have not been arranged about the viewer such that a continuous picture is displayed about the viewer in all viewable directions.
In 1984, U.S. Pat. No. 4,463,380 by John T. Hooks, Jr., marketed by Vought Corporation of Dallas, TX disclosed an image processing system which processes information corresponding to a selected geographical region of a geometric system which is stored and selectively retrieved and processed to reproduce images corresponding to selected portions of the geographical or geometric system. In a preferred embodiment, the information is initially recorded photographically by a camera with semi-spherical coverage. Processing, memory, and control systems are employed for producing video signals to be provided to video display units of semi-spherical coverage, for producing simulated real time images corresponding to the selected portion of the geographical or geometric system. A limitation of Hook's system is that only a system for recording, processing, and displaying semi-spherical images is disclosed, and not a system for recording, processing, and displaying an image of spherical coverage. A system with spherical coverage is desirable in order to increase the realism perceived by the viewer. A further limitation is that the whole process can not be accomplished in near realtime, which is desirable for interactive applications, such as teleconferencing and telerobotic applications.
In 1987, U.S. Pat. No. 4,656,506, by the inventor of the present invention, disclosed the a spherical projection system that rear screen projected a panoramic scene of spherical coverage completely around the viewer. A limitation of the '506 system is that plural cameras used to achieve spherical coverage had limited portability and compactness. A further limitation is that the rear projection screen system required a large amount of space for rear projection and the system does not prescribe a method whereby portions of the images can be transmitted to remote sides of the viewer other than by reflective mirrors.
Since the early 1980's a variety of companies have marketed T.V. special effect generators and image processing systems for accomplishing three dimensional visual effects on a display monitor. Examples of such generators and image processing systems include: The "Digital Production Center" by Quantel Inc. of Stamford, CT, whose Mirage special effects system can perform 3-D image manipulation in realtime with live video (U.S. Pat. No. 4,334,245); and less expensive image processing systems such the "Illuminator" videographics printed circuit board by Matrox of Quebec, Canada, whose board level system operates from of an expansion port of an IBM PC. Both of these systems allow video images to be manipulated and integrated with computer graphics. Prior art has not incorporated such image processing systems to process an image recorded by a panoramic camera of spherical coverage such that a viewer can pan, scroll, and zoom in on portions of the recorded panoramic scene on a HMD system.
Various head mounted display (HMD) systems have been disclosed which display images in front of the viewer's eyes such that the viewer views a virtual reality. To date images for display on HMD systems have been generated by a single camera or two cameras pointing in a single direction or by using computer graphics. A camera system of spherical coverage has not been proposed for use with HMD systems in prior art. Camera systems of prior art have had limited fields of view, not allowing for replay of a scene having spherical coverage. Some prior art HMD systems, such as the "EyePhone" TM by VPL Research Inc., Redwood City, CA, included a position sensing system to electronically define the direction in which the viewer is looking and to define the computer graphic scene to be generated in the HMD system. Makers of such position sensing systems include the Isopace 3D by Polehmus Navigational Systems of Colchester, VT. Prior art has not incorporated position sensing systems to interact with a camera recorded image of spherical coverage. A limitation of interacting with a computer graphic scene is that it typically lacks the realism of a camera recorded scene.
A variety of remotely piloted vehicles (RPV) have been proposed to work with viewing systems of less than spherical coverage. Prior art RPV and unpiloted autonomous vehicles, have not incorporated a camera system with spherical coverage, processing, and display systems of the present invention.